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Summary 


The  goal  of  the  project  was  to  elucidate  computational  mechanisms  underlying  perceptual 
decisions  between  multiple  alternatives  as  well  as  their  neural  implementation.  To  this 
end  we  have  performed  behavioral  experiments  in  humans  as  well  as  combined 
behavioral  and  neurophysiological  experiments  in  non-human  primates.  Computational 
modeling  techniques  were  used  for  addressing  the  computational  mechanisms  underlying 
the  perceptual  decisions.  A  multi-alternative  version  of  the  random  dot  motion  direction 
discrimination  task,  using  a  multi-component  stimulus,  was  found  to  provide  a  rich 
behavioral  dataset,  spanning  accuracy  levels  from  chance  to  perfect  and  a  wide  range  of 
mean  response  times  while  providing  simultaneous  experimental  control  over  how  much 
sensory  evidence  is  provided  for  each  of  the  alternatives.  The  behavior  was  found  to  be 
consistent  with  a  number  of  integration-to-threshold  mechanisms,  including  integration 
with  and  without  leakage,  feedforward  as  well  as  feedback  inhibition  mechanisms  for 
mediating  the  competition  between  the  alternatives,  and  linear  as  well  as  nonlinear 
mechanisms  for  combining  the  signals  associated  with  the  different  alternatives.  While 
these  different  mechanisms  produce  virtually  indistinguishable  decision  behavior,  we 
have  been  able  to  demonstrate  that  the  internal  dynamics  of  these  mechanisms  are  quite 
different,  which  should  be  addressable  by  performing  high-resolution  recordings  of  brain 
activity  while  decisions  are  being  made.  To  this  end  we  have  trained  non-human  primates 
to  perform  the  same  decision  task  and  we  have  performed  multi-electrode  recordings  of 
decision-related  activity  in  parietal  cortex.  Data  collection  and  analysis  are  currently 
being  finalized  to  be  able  to  apply  them  to  discriminating  between  different  decision 
models.  We  have  also  addressed  how  different  decision  mechanisms  relate  to  multi- 
alternative  decision  theory. 


Report 


Human  behavior 

A  multi-alternative  version  of  the  random  dot  motion  direction  discrimination  task,  using 
a  multi-component  stimulus,  was  used  to  address  the  computational  and  neural 
mechanisms  underlying  perceptual  decisions  between  multiple  alternatives.  Briefly, 
subjects  were  asked  to  watch  a  visual  noise  stimulus  with  three  embedded  components  of 
coherent  motion  and  to  indicate  the  direction  of  the  strongest  component.  The  three 
motion  coherences  were  randomly  chosen  from  trial  to  trial  and  the  subjects’  choices  and 
response  times  were  measured  (see  Niwa  &  Ditterich,  2008  for  further  details).  The 
resulting  behavior  spanned  the  full  range  of  accuracy  levels  (from  chance  to  perfect),  a 
wide  range  of  mean  response  times  (from  appro.x.  600  ms  to  approx.  1,600  ms),  and  the 
simultaneous  manipulation  of  how  much  sensory  evidence  was  provided  for  each  of  the 
alternatives  allowed  an  independent  manipulation  of  either  accuracy  or  response  time. 

Computational  models  accounting  for  the  human  behavior 

The  behavior  was  found  to  be  consistent  with  an  integratlon-to-threshold  model  with 
three  integrators  (one  for  each  choice  option),  which  all  accumulate  a  different  linear 
combination  of  three  task-relevant  sensory  evidence  signals.  As  soon  as  one  of  the 
integrators  reaches  a  critical  decision  threshold  the  choice  is  determined  and  the  decision 
process  terminates  (see  Niwa  &  Ditterich,  2008  for  further  details). 

In  the  meanwhile  we  have  been  able  to  demonstrate  that  this  is  only  one  of  a 
number  of  possible  integration-to-threshold  mechanisms  that  could  be  responsible  for  the 
observed  behavior.  The  different  alternatives  include  integrators  with  and  without 
leakage,  feedforward  as  well  as  feedback  inhibition  mechanisms  for  mediating  the 
competition  between  the  integrators,  as  well  as  linear  and  nonlinear  ways  of  combining 
the  integrator  signals.  The  feedfonvard  inhibition  model  published  in  Niwa  &  Ditterich 
(2008)  and  the  feedback  inhibition-based  Leaky,  Competing  Accumulator  (LCA)  model 
(Usher  McClelland,  2001)  are  both  examples  of  decision  mechanisms  relying  on  linear 
signal  combinations,  whereas  the  implementations  of  MSPRT  that  have  been  proposed  by 
Bogaez  &  Gurney  (2007)  and  by  Bogaez  (2009)  make  use  of  nonlinear  signal 
combinations.  While  these  different  mechanisms  are  virtually  indistinguishable  in  terms 
of  the  produced  decision  behavior,  their  internal  dynamics  can  be  shown  to  be  quite 
different.  They  make  different  predictions  how  the  states  of  the  integrators  should  evolve 
over  time  and  also  for  the  correlation  structure  of  the  input  signals  to  the  integrators.  It 
should  therefore  be  possible  to  discriminate  between  these  mechanisms  based  on 
simultaneous  high-resolution  (both  spatial  and  temporal)  recordings  of  decision-related 
brain  activity  associated  with  different  alternatives.  It  is  further  possible  to  address  how 
the  different  decision  mechanisms  are  related  to  multi-alternative  decision  theory.  For 
example,  for  the  simpler  case  of  decisions  between  two  alternatives  it  can  be  shown  that 
the  decision  should  be  made  on  the  basis  of  the  likelihood  ratio  between  the  two 
alternatives  and  that,  if  the  decision  is  based  on  sequential  sampling  of  independent 
sensory  evidence,  the  decision  process  should  be  terminated  when  the  likelihood  ratio 
exceeds  a  particular  threshold.  This  algorithm,  the  sequential  probability  ratio  test 
(SPRT),  has  been  proven  to  guarantee  the  lowest  possible  average  sample  size  (or  mean 
decision  time)  for  any  desired  accuracy  level  (Wald,  1945).  Furthermore,  the  drift 


2 


diffusion  model,  a  popular  mathematical  psychology  model  of  decisions  between  two 
alternatives,  is  an  implementation  of  this  algorithm  if  the  drift  rate  is  proportional  to  the 
logarithm  of  the  likelihood  ratio,  and  the  difference  between  the  activities  of  two  pools  of 
opposing  sensory  neurons  has  been  shown  to  approximate  this  property  (Gold  &  Shadlen, 
2001).  The  situation  is  more  complicated  when  more  than  two  alternatives  are  available. 
In  this  case,  the  optimal  algorithm  for  any  desired  accuracy  level  is  still  unknown.  It  has 
been  shown  that  an  extension  of  SPRT,  the  multi-hypothesis  sequential  probability  ratio 
test  (MSPRT)  is  asymptotically  optimal  in  the  case  of  a  negligible  error  rate  (Dragalin  et 
al.,  1999).  However,  human  decision  makers  are  rarely  perfect.  I  have  extended  this 
framework  to  discuss  what  kind  of  statistical  tests  are  implemented  or  approximated  by 
different  decision  mechanisms  and  how  their  optimality  compares  in  a  biologically 
relevant  range  of  error  rates.  A  manuscript  is  currently  being  finalized  and  will  be 
provided  when  published. 

Neural  recordings  from  non-human  primates 

To  be  able  to  perform  high-resolution  recordings  of  decision-related  brain  activity,  three 
non-human  primates  were  trained  to  perform  the  same  task  as  our  human  observers. 
Although  they  did  not  quite  reach  the  same  accuracy  level  (approx.  80%  in  humans  and 
approx.  70%  in  monkeys)  and  although  they  were  overall  faster  than  the  human  observers 
(mean  response  times  ranging  from  approx.  600  ms  to  approx.  1,000  ms),  the  overall 
behavioral  pattern  was  qualitatively  similar.  Multi-electrode  recordings  were  performed 
in  the  lateral  intraparietal  area  (LIP)  in  parietal  association  cortex.  Neurons  in  this  area 
were  previously  shown  to  carry  decision-related  activity  in  a  motion  discrimination  task 
(Roitman  &  Shadlen,  2002).  ^^enever  possible,  simultaneous  recordings  from  at  least 
two  neurons,  associated  with  either  the  same  or  with  different  choice  options,  were 
performed.  In  addition  to  action  potentials,  local  field  potentials  (LFPs)  were  also 
recorded.  The  basic  firing  rate  pattern  of  the  recorded  neurons  turned  out  to  be  consistent 
with  previous  observations:  approx.  200  ms  after  motion  onset,  the  firing  rate  shows  a 
coherence-dependent  ramping  behavior  (see  Fig.  1  for  preliminary  data)  and,  when  the 
neuron  is  associated  with  the  chosen  option,  reaches  a  stereotyped  level  immediately 
prior  to  the  monkey’s  behavioral  response  (see  Fig.  2). 


Ttme  with  respect  to  onset  of  motion  stimulus  frr^ 
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Since  we  have  used  a  3-component  stimulus,  the  slope  of  the  initial  ramp  (between  250 
and  400  ms  after  motion  onset),  which  is  assumed  to  reflect  the  accumulation  of  sensory 
evidence,  is  expected  to  be  a  function  of  the  three  relevant  coherence  levels.  The 
difference  between  the  coherence  of  the  component  providing  evidence  for  picking  a 
choice  target  inside  the  recorded  neuron’s  response  field  and  the  average  of  the  other  two 
coherences  turned  out  to  be  a  major  determining  factor  (see  Fig.  3).  This  is  what  the 
model  published  in  Niwa  &  Ditterich  (2008)  would  predict.  However,  there  are  two  other 
interesting  features;  The  slopes  have  a  clear  positive  offset  (they  are  not  zero  for  zero  net 
coherence)  and  they  can  be  quite  different  for  the  same  net  coherence  (see  the  three 
symbols  stacked  on  top  of  each  other  at  zero  net  coherence).  The  firet  effect  is  not 
predicted  by  a  feedforward  inhibition  model  with  perfect  integrators  and  Churchland  et 
al.  (2008),  who  have  also  reported  this  phenomenon,  have  interpreted  it  as  evidence  for  a 
top-down  soft  deadline  signal.  However,  as  discussed  in  Ditterich  (2006),  such  a  signal 
would  be  expected  to  have  a  strong  impact  on  the  shape  of  the  response  time  (RT) 
distributions.  Our  preliminary  data,  however,  do  not  provide  any  strong  evidence  for  a 
major  shortening  of  the  tail  of  the  RT  distributions.  Instead,  preliminary  modeling 
indicates  that  this  offset  could  just  be  the  consequence  of  the  nonlinear  properties  of  a 
biological  integrator:  it  cannot  represent  negative  values.  The  second  observation  (quite 
different  slopes  for  the  same  net  coherence)  is  also  interesting  because  it  could  be  an 
indication  of  a  normalization  process  operating  at  the  level  of  the  decision  circuitry.  We 
are  currently  looking  into  this  in  more  detail.  The  three  symbols  stacked  on  top  of  each 
other  at  zero  net  coherence  belong,  from  top  to  bottom,  to  stimuli  with  no  coherent 
motion,  with  coherent  motion  of  10%  in  each  direction,  and  with  coherent  motion  of  20% 
in  each  direction.  Thus,  the  slope  is  actually  decreasing  with  increasing  coherence.  The 
slope  change  therefore  also  cannot  be  explained  by  a  feedback  inhibition  model,  which 
would  predict  an  effect  in  the  opposite  direction  (higher  coherence  should  be  providing  a 
stronger  excitatory  drive  to  the  integrator). 
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Fig.  3:  Firing  rate  slopes  as  a 
function  of  net  coherence.  The 
color  of  the  symbols  indicates  the 
sum  of  all  three  coherences  (blue  = 
low,  red  =  high). 


As  mentioned  above,  different  decision  mechanisms  make  different  predictions  for  the 
correlation  structure  of  the  input  signals  to  the  integrators.  Our  hope  was  that  this 
correlation  structure  would  show  up  in  the  correlation  structure  of  the  spike  pattern 
emitted  by  two  simultaneously  recorded  neurons  that  are  coding  for  two  different  options. 
Unfortunately,  the  majority  of  pairs  of  neurons  that  code  for  two  different  options,  which 
we  have  recorded  so  far,  did  not  show  any  significant  correlation  in  their  spike  timing. 
This  can  be  interpreted  a  number  of  ways  and  only  more  detailed  modeling  can  help  us 
come  up  with  an  answer.  Possible  options  include  decision  mechanisms  that  actually 
predict  almost  no  correlation  between  the  input  signals  to  the  integrators,  or,  since  we  are 
only  recording  two  individual  neurons  out  of  probably  quite  large  pools,  the  connections 
could  be  so  sparse  that  correlations  that  should  be  there  at  the  population  level  cannot  be 
seen  at  the  single  neuron  level.  We  have  also  recorded  from  pairs  of  neurons  that  code  for 
the  same  option.  The  majority  of  these  show  the  expected  positive  cross-correlation  near 
zero  lag  (due  to  common,  probably  excitatory,  input),  but  they  also  tend  to  show  an 
unexpected,  asymmetric  negative  cross-correlation  at  a  relatively  large  lag  of  approx. 
10  ms  (see  Fig.  4  for  a  typical  example).  This  could  indicate  unexpected,  directed 
inhibitory  connections  between  neurons  belonging  to  a  particular  decision  pool,  and  we 
are  currently  looking  into  this  finding  in  more  detail. 
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Fig.  4:  Typical  example  of  the 
spike  time  cross-correlation 
structure  between  a  pair  of  LIP 
neurons  coding  for  the  same  choice 
option 


In  addition  to  spiking  activity  we  have  also  recorded  local  field  potentials  (LFPs).  There 
is  still  considerable  debate  what  exactly  LFPs  are,  but  there  is  some  consensus  that  they 
are  strongly  influenced  by  dendritic  activity  and  therefore  might  be  considered  a  measure 
of  the  input  to  a  brain  area  in  contrast  to  the  spiking  activity,  which  reflects  the  output  of 
the  recorded  neuron.  We  are  still  in  the  process  of  analyzing  the  LFPs,  but  preliminary 
results  indicate  that  they  contain  information  about  the  motion  stimulus,  that  they  do  so 
before  the  spiking  activity  shows  selectivity  for  the  motion  stimulus,  and  that  what  they 
are  coding  does  not  seem  to  be  redundant  with  the  information  in  the  spiking  activity. 
Fig.  5  shows  the  difference  in  the  low-frequency  power  spectrum  between  trials  with  only 
low  coherences  and  trials  with  only  high  coherences.  (This  is  where  we  see  the  strongest 
effect  in  the  LFPs  in  contrast  to  the  firing  rate,  where  the  strongest  effect  is  seen  between 
trials  with  strong  support/weak  evidence  against  and  weak  support/strong  evidence 
against.)  The  main  effect  is  a  reduction  in  the  LFP  power  around  8  Hz  for  overall  high 
coherence  trials  in  the  first  200  ms  after  motion  onset.  We  are  currently  performing  a 
more  detailed  quantitative  analysis  of  this  phenomenon. 


Time  wrt  motion  onset  [s] 


Fig.  5:  Difference  in  LFP  power  between 
trials  with  only  low  coherences  and  trials  with 
only  high  coherences.  Stars  indicate 
time/frequency  combinations  where  the  95% 
confidence  intervals  of  both  spectra  do  not 
overlap. 
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We  are  currently  in  the  process  of  finishing  up  the  neural  recordings  and  their  analysis. 
We  still  need  to  combine  the  neural  data  and  the  modeling.  Some  preliminary  results 
have  been  presented  at  last  year’s  Society  for  Neuroscience  Annual  Meeting  (Bollimunta 
&  Dilterich,  2009).  More  results  will  be  presented  this  year  and  publications  will  be 
provided  as  they  become  available. 
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